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Abstract 

We consider a population of agents that are heterogeneous with respect to (i) their strat- 
^ I egy when interacting rig times with other agents in an iterated prisoners dilemma game, (ii) 

their spatial location on K different islands. After each generation, agents adopt strategies 
proportional to their average payoff received. Assuming a mix of two cooperating and two 
defecting strategies, we first investigate for isolated islands the conditions for an exclusive 
domination of each of these strategies and their possible coexistence. This allows to define a 
^ , threshold frequency for cooperation that, dependent on Ug and the initial mix of strategies, 

describes the outbreak of cooperation in the absense of migration. We then allow migration 
i-C . of a fixed fraction of the population after each generation. Assuming a worst case scenario 

where all islands are occupied by defecting strategies, whereas only one island is occupied by 
cooperators at the threshold frequency, we determine the optimal migration rate that allows 
the outbreak of cooperation on all islands. We further find that the threshold frequency 
divided by the number of islands, i.e. the relative effort for invading defecting islands with 
cooperators decreases with the number of islands. We also show that there is only a small 
bandwidth of migration rates, to allow the outbreak of cooperation. Larger migration rates 
destroy cooperation. 

keywords: migration, cooperation, iterated prisoners dilemma 



1 Introduction 

Human Migration, i.e. the movement of large numbers of people out of, or into specific geo- 
graphical areas, is seen as one of the biggest challenges that face the human societies in the 21st 
century. On one hand, part of the human population has reasons to emigrate into countries which 
provide a "better" life - on the other hand, industrialized countries cannot sustain their current 
situation without the immigration of people. The real problem arises because the "demand" and 
the "supply" side cannot be matched. Industrialized countries fear that immigrants do not con- 
tribute to their further economic growth but, on the contrary, deplete their wealth by taking 
advantage of a social security, health, and educational system which they did not contribute to. 
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If we move this problem on the more abstract level of a game-theoretical model, we can distinguish 
between two types of agents: those cooperating, i.e. being able to integrate in a society and to 
contribute to a common good, namely economic growth, and those defecting, i.e. without the 
ability to socially integrate and thus depleting a common good at the cost of the cooperating 
agents. Certainly, based on their past experience, agents can adapt, i.e. they can change their 
strategy from defection to cooperation and vice versa dependent on the payoff they receive in a 
given environment. The question for an industrialized country would be then to define an optimal 
immigration rate that (a) does not destroy the common good, and (b) allows agents to adapt to 
the assumed cooperative environment within one or two generations, even if they may have not 
immigrated with a cooperative strategy. 

The problems of cooperation and defection and the payoff-dependent adoption of strategies have 
been discussed in the framework of the Prisoner's dilemma (PD) and the iterated PD (IPD) 
game (see Sect. [2]). With our paper, we add to this framework the ability to migrate between 
different countries ("islands") . Our aim is to reveal optimal conditions for the migration of agents 
such that cooperating strategies can take over even on those islands where they were initially 
not present. 

We note that migration was previously studied in a game-theoretical context by different authors 
0, 1^. Our work differs from these attemts in various respects. First of all, we do not assume 
that migration is based on the anticipated success 0, S| ~ this shifts the conceptual problem 
of the "outbreak of cooperation" to proposing rules with non-local information such that two 
cooperators meet at the same place, from which cooperating clusters can grow. We also do not 
make migration dependent on local variables such as the number of defectors in the neighborhood 
a or random choices of "empty" places BB. In fact, human migration is rarely targeted at less 
densely crowded places, it is rather the opposite. Further, we do not assume one-shot games such 
as the PD, but instead consider the IPD in which the number Ug of repeated interaction as well 
as the mix of up to 8 different strategies plays a crucial role. 

Eventually, we do not use an agent-based model in which update and migration rules are freely 
defined, to study their impact on computer simulations on a lattice. Our approach proposes a 
population based model in which subpopulations are defined with respect to (a) their interaction 
strategy, and (b) their spatial location. The consideration of separated "islands" allows a coarse- 
grained implementation of spatial structures which is in between a lattice or network approach 
and a meanfield description. It is known that spatial structures have an impact on the outbreak 
of cooperation 0, [l3, 14 1, but their infiuence varies with other degrees of freedom, such as update 
rules, synchronization, interaction topology, payoff matrix. 

Therefore, in this paper we adopt mostly standard assumptions about the interaction type (IPD 
with Ug encounters) and interaction topology (panmictic subpopulations), strategy adoption 
(replication proportional to payoff), and migration (fixed fraction of the population). To under- 
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stand the basic dynamics, we first investigate "isolated" islands (no migration) to find out about 
the conditions for the "outbreak of cooperation" without external influence. This "outbreak" is 
defined as the critical point (strategy mix, number of encounters Ug) beyond which a whole island 
is going to be occupied by cooperating agents, if agents adopt strategies proportional to their 
average payoff. Then, we add migration between islands to this dynamics to find out under which 
conditions the outbreak of cooperation can be enhanced. It is important to note that migration 
does not distinguish between strategies or islands, i.e. there are no better suited strategies for im- 
migration, or bad places with high emigration rates - which we consider as artificial assumptions 
to explain the "outbreak of cooperation". 

To determine the robustness of our findings, we always consider worst case scenarios, i.e. initial 
settings in which most islands have either an entirely defective subpopulation, or at least a 
defective majority. We further control for important parameters such as the pool of available 
strategies s, the number of interactions Ug or the number of islands K, for which critical conditions 
are derived. Our finding that migration is indeed able to boost the outbreak of cooperation is 
remarkable both because it is based on minimal assumptions about the dynamics and because 
it follows from a quite systematic investigation of the underlying conditions. 

2 Rules of the game 
2.1 Strategic interaction 

Let us investigate a population of N agents divided into subpopulations on K different islands 
which imply a coarse-grained spatial structure, i.e. N = "^Zk -^k- Agents at island k are assumed 
to interact with the Ni^ — 1 other agents on their island {panmictic population). But, in general 
we assume that migration between the islands is possible, with the respective migration rates 
given by m„„. Figure [1] shows the case of 3 different locations. 

Our model basically considers two different time scales for interaction and migration. We define a 
generation G to be the time in which each agent has interacted with all other iV^ — 1 agents a given 
number of times, denotes as Ug. Thus the total number of interactions during each generation 
in the panmictic population is roughly iV|/2 x Ug. At any given time, agent i can interact only 
with one other agent in a "two-person game". But during one generation G, it interacts with all 
other Nf: — 1 agents Ug times (repeated two-person game). 

For the interaction, we adopt the well known iterative prisoner's dilemma (IPD) from game 
theory. At each encounter between agents i and j, both have two actions to choose from, either 
to collaborate (C) or to defect (D), without knowing the action chosen by their counterpart. The 
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Figure 1: Subpopulations Nf^ (with different strategies) are spatially distributed on k islands 
from where they can migrate at rates rriuv 

outcome of that interaction is described in the following payoff matrix: 

C D 

C {R,R) iS,T) (1) 
D iT,S) {P,P) 

If both agents have chosen to collaborate, then they both receive the payoff R. If one of the agents 
chose to defect and the other one to cooperate, then the defecting agent receives the payoff T and 
the cooperating the payoff S. If both defect, they both receive the payoff P. In the special class 
of Prisoner's Dilemma (PD) games, the payoffs have to fullfil the following two inequations: 

T>R>P>S] 2R>S + T (2) 

which are met by the standard values T = 5, R = 3, P = 1, S = 0. This means in a cooperating 
environment, a defector will get the highest payoff. 

For Ug = 1 ("one-shot" game) choosing action D is unbeatable, because it rewards the higher 
payoff for agent i no matter if the opponent chooses C or D. At the same time, the payoff for 
both agents i and j is maximized when both cooperate. A simple analysis shows that defection 
is a so-called evolutionary stable strategy (ESS) in a one-shot PD. If the number of cooperators 
and defectors in the population is given by N'^ and N"^ = N — N'^ respectively, then the expected 
average payoff for cooperators will be a'^ = [Ry. N'^ + S x N^)/N. Similarly the expected average 
payoff for defectors will be a"^ = (T x iV^ P x N'^)/N. Since T > R and P > S, a'^ is always 
larger than for a given number N'^, and pure defection would be optimal in a one-shot game. 
Even one defector is sufficient to invade the complete population of — 1 cooperators. 
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But in a consecutive game with short memory, both agents, by simply choosing D, would end 
up earning less than they would earn by cooperating. Thus, the number of games Ug two agents 
play together becomes important. This makes sense only if the agents can remember the previous 
choices of their opponents, i.e. if they have a memory of 1^ ng — 1 steps. Then, for the iterated 
Prisoner's Dilemma (IPD), they are able to develop different strategies based on their past 
experiences with their opponents. We note that the IPD game was studied both in the context 
of a panmictic population [lol . ^2! . Esl ) and assuming a spatial population structure 



01 a panm] 

3 HQ. 



Usually, in an IPD only a one-step memory is taken into accout. Based on the known previous 
choice of its opponent, either C 01 D, agent i has then the choice between eight different strate- 



gies. Following a notation introduced in these strategies are coded in a 3-bit binary string 
which always refers to cooperation. The first bit represents the initial choice of agent i: 
it is 1 if agent i has cooperated, and if it has defected initially. The two other values refer always 
to the previous choice of agent j. is set to 1 if agent i chooses to cooperate given that agent 
j has cooperated before and otherwise. I'^ is similarily set to 1 if agent i chooses to cooperate 
given that agent j has defected before and otherwise. 

Both I'^ and I'^ can be also interpreted as probabilities to choose the respective action given the 
knowledge of the previous choice of the opponent, i.e. {0 < Ip,I'^ < 1} in the stochastic case. 
In the deterministic case considered in this paper, I'^ and /"^ are either or 1. Out of the eight 
possible strategies, to keep our results in a tractable manner, we consider the following four 
strategies z^*) (s = 1,2,3,4): 



(1) : 


[1|10] 


TFT 


(2) . 


[0|00] 


ALL-D 


(3) . 




ALL-C 


(4) . 


[0|01] 


A -TFT 



(3) 



All eight strategies have been discussed in [l5i| , in this paper we have dropped those variants that 
differ only in the first move but then continue as described in the following. The strategies ALL-D 
and ALL-C are obvious: for z^^) agent i always chooses to defect regardless of the previous choices 
of agent j, 2*^^^ follows likewise. These two strategies represent the limit case of no memory, i.e. 
agent i is simply "fixed" as a cooperator or a defector. 

The most interesting strategy z^^^^ known as "tit for tat" (TFT), means that agent i initially 
cooperates and continues to do so, given that agent j was also cooperative in the previous move. 
However if agent j was defective in the previous move, agent i chooses to be defective, too. 
This strategy was shown to be the most successful one in iterated Prisoners Dilemmas If 
initially two cooperative agents meet, they stay cooperative (forever, in a deterministic game), 
this way maximizing their average payoff. This encourages other agents to adopt TFT. Hence, 
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the "outbreak of cooperation" in a community of N agents occurs as the amphfication of a 
cooperative initial fluctuation. 

The fourth possible strategy z^^^ is just the negation of the most successful one, therefore named 
Anti-TFT. It means that agent i starts as a defector and continues to be so, as long as it meets 
with cooperative agents j, this way receiving the highest possible payoff T. But, if it meets an 
agent that has previously defected, it changes its "behavior" to cooperation. This makes some 
sense, since the loss in payoff is not that much - P would be just slightly higher than S. On the 
other hand, if the colloborative agent meets again with another cooperative agent, the payoff 
would be R, which is a much higher gain. 



2.2 Dynamics of replication and migration 

In this paper, we are interested in how the frequencies of the different strategies would evolve 
in the agent community. The fraction of agents choosing strategy z^*^ on island k is defined 
as fl^\t) = N^^\t)/Nk, where the different strategies are given by eq. ([3]). Similarly, the total 
fraction of agents playing strategy z^*) in the whole system, is given as f^'^^t) = ^ Y2k /i^^C^) ~ 
N^'^\t)/N . For the dynamics we may assume that during each generation G every agent behaves 
according to a fixed strategy picked up from the pool of possible strategies. This strategy can 
be changed only after one generation is completed. The "update" rule for the strategy is simply 
given by the success of different strategies during the past generation. As the evaluation criteria, 

(s) 

we choose the average payoff aj, , each strategy has received during the last generation on island 
k. This will be compared with the total average payoff Ok that gives an estimate of the overall 
dynamics on that island: 

-Mt) = i2-l'\t)ft\t); j2ft\t) = l (4) 

S = l 3 = 1 

For the evolution of the frequencies of the different strategies, the following dynamics is postu- 
lated: 

ft\G+^) = ^i^ft\G) (5) 

It means that in the next generation G + 1 the share of agents choosing a particular strategy 
z^^') has grown/shrunk - i.e., strategy s has replicated - according to the relative performance 
of this strategy on island k during the previous generation G. In population dynamics, this is 
known as fitness-proportional selection [7j|, we adopt it here since the relative performance of a 
certain strategy can be also interpreted as its fitness. 

The dynamics of eq. ([5]) does not take into account that agents can migrate between the K 
islands. For simplicity, we assume that each island k is occupied by the same number of agents 
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A^fc = N/K and migration can occur between between any two islands with a constant and equal 
migration rate niuv = m/{K — 1). We interpret m € {0, 1} as the total fraction of the agent 
population at any given island that migrates to other islands in a given time interval, namely one 
generation. In a first approximation, we assume that migration only occurs after a generation is 
completed, i.e. at fixed times G, G + 1. 

While the fraction m is fixed (but controlled for, afterwards), the composition of the migrating 

subpopulation across the existing strategies s is not. We assume this composition to be pro- 
fs) 

portional to the fraction fj^ of these strategies at island k at the time when generation G is 
completed and migration occurs, to affect the dynamics during the next time interval G + 1. This 
means migration changes the fraction fjf\G + 1) by an additional amount T representing the 
difference between the influx and the outflux of the agents playing strategy z^'^^: 



K 



(6) 



which defines the complete dynamics by the following set of iterative equations: 

&\g + 1) = ^^!j§l&\G)+4'\G) (7) 



Using J-{0) = and /^(O) = i.e. an equal share of s strategies on each island as initial 
conditions, the dynamics is completely determined if we know the respective average payoffs 
which are derived in the following section. 

2.3 Determining the payoffs of repeated interaction 

In a deterministic game, we are able to calculate the average payoff af''^^ that is received by 
agent i playing strategy z^'^\ q G s, Ug times with an agent j playing strategy z^^\ r G s. For 
s = 1,2,3,4, the results are given in the 4x4 payoff matrix of eq. ([8]). Note that the matrix is 
not symmetric, since it gives the average payoff of agent i. r refers to the column and q to the 
row of eq. 



n„ 



UgR 



S + (Ug 



1)P UgR 



T+{ng-l)P 

UgR 

a(4i) p + {^ng-l)S UgT 



UgP 
UgS 



UgR 



,(14) 



HgT P + {Ug 



1)T 



UgS 

n.(44) 



(8) 
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where 

(ng rem 4) (rig rem 4) 

a(i4) = {ng diY 4) V+ ^k] a(^i) = (ngdiv4)P+ ^ 

k=l k=l 
(rig rem 2) 

a(44) = {ngdiY2){P + R)+ ^ 4 (9) 

k=l 

The symbols T, R, P and S refer to the payoff matrix of the 2-persoii game, eq. ([1]). Further, we 
have used Ug = {ngdivy)y + (rigremy), with < (n^remy) < y. (n^divy) means the integer 
part of the division Ug/y, i.e. (ngdiv4) = 4 for ng = 18; while (n^remy) means the remainder, 
i.e. (ngrem4) = 2 for ng = 18. The constants appearing are defined as follows: 

V = T + R + P + S- bi = S; b2 = P; h = T; b4 = R 

ci = T; C2 = P; 03 = 8; 04 = R di = P; d2 = R (10) 

To verify how the matrix entries of eq. ^ are derived we have provided a number of illustrative 
examples shown in [XI 

With the known average payoff resulting from each possible interaction, the average payoff per 
strategy at island k is simply given as: 

^^=i:«^^^vr; Efk^=^ is=i,..A) (11) 

r=l r=l 

where fj^^ denotes the fraction of agents playing strategy z^'^'^ at island k and the a^*"*^ are given 
in eq. ([S]). Stricly speaking, eq. (|11|) has to consider the fact that an agent does not play against 
itself. This would lead to a correction term of the order of a^^^^ /N, which is small and therefore 
neglected here. 

2.4 Calculating the fix points 

With the above specifications we are now able to solve the dynamic equations with respect to the 
fraction of agents fjf\t) playing strategy z^'^^ Here, we are mostly interested in the impact of 
migration on the prevailence of certain strategies, therefore we will discuss first the case without 
migration, eq. ([5]), which should be used as a reference for the case with migration, eq. jT]). Also, 
we are mostly interested in the stationary solutions of the respective dynamics, i.e. in the fix 

(s) 

points reached after a suffiently large number of generations. Our focus is then on three 
issues which crucially determine the long-term outcome: 

(a) the initial frequencies of strategies f^'^\0), 
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(b) the number of repeated interactions, Ug, as this enters the payoff matrix, eq. ([8]) and therefore 

affects the average payoff per strategy, eq. (fTTj) 

(c) the impact of the migration rate m on the prevaihng strategies. 

(s) 

The stationary values of fj^ are reached if the different frequencies do not change anymore in 
the new generation. This leads to the stationary condition: 

fjf\G + l) = fjf\G) = fi'^ (12) 

where fjf'^ shall denote the stationary (i.e. asymptotic) frequency of strategy s, in contrast to 
fjf\o) that denotes the initial frequency. For the full dynamics which includes migration, eq. ([7]), 
we solve this condition numerically. But for the case without migration, eq. ([5]), the stationary 
condition leads to Ofc/^^^ = Ci^^^ fk^\ which reduces to the quite simple expression = a^j^^ only 
if / 0. 

To further discuss the case of isolated islands, we drop the index k. For m = eq. (jl2p with eq. 
([5]) basically yields two different types of solutions, which could be either stable or unstable: 

(i) the exclusive dominance of any one strategy z^^^ {q S s) while all other strategies z^'^\ 
r ^ q, have disappeared, i.e. 

/(?) = 1; flfc = a(«) and f^"^^ =0 for r^q (13) 

(ii) the co-existence of some (or all) of the strategies z^'^^ , z^'^^ with the same average payoff a, 
but probably different frequencies of agents, f^'^\ f^^\ i-e. 

4 4 

= (14) 

i=l 1=1 

In the deterministic case considered here it solely depends on the initial conditions f^'^\G = 0), 
which of the possible stationary solutions will be eventually reached. After we know the attractor 
basins, i.e. the range of possible initial conditions that lead to a given stationary solution and the 
separatrix deviding them, we could tell from the outset which strategies will be adopted by the 
population, and which strategies will disappear. Therefore, in the next section we will calculate 
the attractor basins for the case of isolated islands, to compare these later on for the case of 
migration which, as we will see, shifts these basins considerably. 
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3 No migration: Coexistence and dominance of strategies 

For isolated islands, condition ()12p leads to s = 4 coupled equations, which follow from eq. ^ 
with eq. (|lip . In addition, we have the boundary condition Yls f^'^^ ~ which leads to three 
independent variables f^^\ f^'^\ f^^^- The separatix then appears as a two-dimensional plane in 
this three-dimensional space, which will be hard to calculate analytically. In order to elucidate 
this multi-dimensional problem, we discuss in|B]the case of only two possible strategies, where 
the separatrix appears as a point in the one-dimensional space, and inOthe case of three possible 
strategies, where the separatrix appears as a line in the two-dimensional space. Here, we continue 
with the full problem of four possible strategies. 

Compared to [Cl strategy z^^^ = A-TFT is added which in the long run benefits agents playing 
ALL-D and therefore will decrease the attractor basin of the cooperating strategies. I.e. consid- 
ering A-TFT as an additional strategy will create a more difficult environment for the invasion 
of cooperation, therefore it will be very interesting to see, under what conditions cooperation 
can survive also in the worst case. The stationary solutions for the frequencies Z^^-* result from 
the following set of coupled equations: 



s=l s=l \r=l J 



a /W _ /(I) + «(*2) f(2) ^ ^(.3) ^(3) + ^(.4) fi4) \ ^ g = 1, 2, 3, 4) ^^^^ 



^(1) + fi2) ^ ^(3) + ^(4) ^ 1 

The matrix elements a^'"'?-' can be calculated from eq. ([5]). With = 4 we find the following 
stationary solutions: 



(i) /(I) = 1, /(2) = 0, = 0, = (stable) 

/(I) = 0, = 1, /(3) = 0, = (stable) 

/(I) = 0, = 0, = 1, = (unstable) 

/(I) = 0, = 0, = 0, /(^) = 1 (unstable) 

{ii) fill = 0.2, fg^ = 0.8, = 0, = (for = 4) (unstable) 

/(I) = 1.0 - (0 < /(3) < 1), /(2) = 0, = (stable) 



(16) 



We note that the stationary solutions also cover the results of the two-strategy case, [B] and the 
three-strategy case, O We find solutions of type (i) where one strategy is exclusively dominating 
the whole population. However, the solution of all agents playing z^^^ = A-TFT is also an unstable 
attractor, because a small perturbation by agents playing either strategy ALL-D or TFT will 
lead to the invasion of that respective strategy into the whole A-TFT population. To see this, we 
also refer to the payoffs received in the respective interactions, eqs. ()2ip . ()22p . From the solutions 
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(ii) that describe the coexistence of strategies we further see that the addition of strategy z^^^ = 
A-TFT does not lead to new coexisting states compared to the case of two strategies, [B] and 
three strategies, [Cj There is only a stable coexistence between strategies z^^^ = TFT and z^^'^ = 
ALL-C. 

The separatrix that distinguishes between the different attractors is in the given case a two- 
dimensional plane in the three-dimensional space of independent frequencies with /^-^^ = 1- 
With respect to the cooperative strategy TFT it defines the threshold frequency f^^^ = f^^\G = 
0) that has to be reached initially, in order to allow the system to converge to a fully cooperative 
state, f^^\G — 7- oo) = 1. Because of the ng-dependence of the payoff matrix, eqs. ([8]), (|24p 
strongly decreases with the number of encounters n^, as shown in Fig.[2l In agreement with known 
results from evolutionary game theory , for a sufficiently large number of interactions the 

whole population will adopt the cooperative strategy, this way increasing the average payoff. The 
important point to notice here is the quantitative analysis which allows us to determine, for a 
given finite number of n^, the threshold initial frequency of agents playing TFT, in order 
let cooperation invade the whole population. Or else, for a given initial frequency /^^^(O), we are 
able to determine the minimum number of interactions, n™°, to let the whole population adopt 
the TFT strategy. 




Figure 2: Threshold frequency f^^ = f^^\0) of agents playing strategy z^^^ = TFT vs. number 
of interactions rig between each two agents, in the case of four strategies. For f^^\0) > f^^^ 
the TFT strategy will be adopted by the whole population, while for /^^^(O) < the z^^^ 
ALL-D strategy will be adopted by the whole population. 



thr' 



To get some further insights, we find it more convenient to present two-dimensional graphical 
projections of this three-dimensional space, where the separatrix naturally appears as a line. The 
most interesting projection is the f^^^ — f^"^^ projection, since it allows us to compare the results 
with the previous cases. The results are shown Fig. [3j They have been obtained by numerically 
solving eq. ^ for the full range of initial frequencies, < f^^\Q) < 1. In the stationary limit, 
the average payoff was then evaluated to find out to which attractor basin the solution belongs. 
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A 1 

/ 


B \ 



0.1 0.2 0.3 0.4 0.5 
f"(0) 




Figure 3: Attractor basins, i.e. range of initial frequencies /^'*^(0) that lead to a particluar stable 
solution, eq. (jl6p . in the case of four strategies. A: adoption of z^^^ = ALL-D strategy in the 
whole population, B: adoption of z^^^ = TFT strategy in the whole population, C: coexistence 
of both = TFT and z^^) = ALL-C strategies. f^^\0) = 1 - f^^^O) - f^'^^O), Ug = 4. (left) 
/(^)(0) = 0.2, (middle) /(4)(0) = 0.5, (right) /W(0) = 0.7. 



The calculation of the attractor basins in Fig. [3] are done for a fixed number of interactions, 
Hg = 4, but for a varying initial frequency of agents playing strategy z^^'^ = A-TFT. Of course, 
this sets limits to the maximum initial frequencies /(^)(0) = l-/(3)(0) and /(2)(0) = l-/(3)(o), 
therefore in Fig. [3] we have scaled the /(*)(0)-axes by the maximum possible value, which should 
be noticed when comparing the figures. For a small initial frequency of A-TFT (Fig. [3] left) we 
find the picture rather similar to the case of three strategies shown in [Cl however the separation 
line between the attractor basins A (defection) and B + C (cooperation) is now a non-linear 
function, different from eq. ([32]). 

With an increasing initial frequency of A-TFT, we find that the attractor basin of defection 
is naturally decreasing in absolute size, but it is increasing relative to the attractor basin of 
cooperation. That means that an increase in the fraction of agents initially playing A-TFT 
always benefits the adoption of the ALL-D strategy, at the end. A similar conclusion can be 
drawn also for the number of agents initially playing ALL-C. We notice that with an increasing 
initial frequency of A-TFT region C, that describes the coexistence of both strategies TFT and 
ALL-C, ceases to exist, i.e. strategy ALL-C has vanished for the benefit of strategy ALL-D. This 
can also be understood by looking at the average payoff matrix, eq. ([8]). If an agent playing 
ALL-D meets with an agent playing ALL-C, it gets the maximum payoff, T, while it gets almost 
the maximum payoff {P + {rig — \)T)/ng when it meets an agent playing A-TFT. Thus, strategy 
ALL-D will eventually invade the population of agents playing either ALL-C or A-TFT, or, in 
other words, the presence of strategies like ALL-C and A-TFT in the population helps strategy 
ALL-D to invade. 

Eventually, we have also calculated numerically the relative size of each attractor basin as shown 
in Fig. m for two different values of = 4 and — ?■ cx). Here the relative sizes a, 6, c and d 
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refer to the three-dimensional space of aU possible initial conditions (which is different from Fig. 
[9] where they refer to the areas A, B and C shown, thus eq. (|33p does not apply here). When 
Hg = 4, the ALL-D basin 'a' is slightly bigger than the cooperative basin 'd' as shown in contrast 
to figure [TOl With ng oo, the size of the ALL-D basin 'a' becomes pretty small as compared 
to cooperative basin 'd'. 



N 
03 

O 




abed 



Figure 4: Relative size of the attractor basins, 'a' represents the size of the ALL-D basin, 'b' rep- 
resents the size of the TFT basin, 'c' represents the size of TFT+ALL-C basin and 'd' represents 
the size of cooperative basin (i.e. b+c). The left bars (shaded area) refer to Ug = 4, while the 
right bars (black area) refer to Ug — > oo. Thus, the change indicates the influence of Ug on the 
size of the attractor basins. 

To conclude this analysis, we have shown for the case of isolated islands (no migration) what 
are the critical initial conditions for cooperative strategies, such as ALL-C, TFT to be adopted 
by the whole agent population. Dependent on the number of interactions n^, we can tell the 
minimum initial frequency of TFT, /'•^^(O) that has to be reached to ensure an cooperative 
outcome dependent on the initial mix of strategies (details of the two and three strategy case 
are provided in [Bl [C]). In the limit of small n^, while one may naively assume that an intial 
increase of the ALL-C strategy benefits cooperation, at the end, this analysis shows that with 
the involvement of A-TFT it eventually benefits defection in the system, which is an interesting 
finding by itself. In the following, we will investigate how this picture changes if we add the 
possibility of migration. 



4 Migration: The rise and fall of cooperation 
4.1 The role of migration 

The previous section has shown that an increase of the number of interaction Ug always supports 
the prevailence of cooperating strategies, such as ALL-C and TFT. Therefore, in the following 
we will concentrate the analysis on the critical conditions which occur for small Ug. We fix = 4 



F. Schweitzer, L. Behera: 
Optimal migration promotes the outbreak of cooperation in hetergeneous populations 

ACS - Advances in Complex Systems (2012) 



(note that = 3 is the threshold number of interaction to ahow the rephcation of cooperating 
strategies at ah, see [B] and Figure [2l 




Figure 5: Migration between the three islands enhances the outbreak of cooperation, (i) Initial 
state (G = 0) using two strategies. The defective strategy ALL-D has the majority on all islands, 
(ii) Final state {G oo) (a) without and (b) with a small migration rate. In (a) only one island 
has adopted cooperation, in (b) all three islands have adopted cooperation 

To understand how migration affects the existing equilibrium states, let us start with the simple 
example shown in Fig. [5J We take the setup of Fig. [T] and consider only two strategies, z^^^ =TFT 
and z(^) =ALL-D, but different initial conditions on the 3 islands. As the detailed analysis of [B] 
has shown, below an initial frequency f^^\0) = 0.2 the only stable stationary state is /^^^ = 0, 
= 1. Hence, if no migration is possible, with the given initial conditions shown in Fig. [5ji) the 
three islands converge into a final state shown in Fig.[5]^iia) where ALL-D prevails in two islands, 
while in one TFT dominates because that island started from an initial condition above the 
threshold. If we however introduce a rather small migration rate of m/2 = 0.04 all three islands 
are eventually dominated by the cooperating strategy, TFT, as shown in Fig. [Sjiib). Hence, even 
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with a rather smaU advantage on one island {f^^\0) = 0.3 is just above the threshold of 0.2), a 
small migration was able to induce a transition toward TFT instead of a relaxation into ALL-D 
on all islands. 

It is precisely this kind of phenomenon that we would like to understand better regarding the 
critical conditions involved. In particular, we concentrate on the impact of the migration rate on 
the threshold value of TFT, to still observe the outbreak of cooperation. The results are found 
numerically, by iterating the set of eqs. ([7]), for = 4 and K islands that have the same total 
number of agents, but different initial strategy distributions. 




G G G 



Figure 6: Evolution of the average payoff in the K = 5 island model for different migration rates 
m: (left) m = 0.0, (middle) m = 0.08, (right) m = 0.2. Parameters: Panmictic population in 
K = 5 island model consisting of TFT and ALL-D. Initial frequencies of TFT: f[^^ = 0.3, /g^^ = 
0.25, /i') = 0.1, /i') = 0.18, /g^') = 0.15 and /f = 1 - fi'\ 

Fig. [6] shows results of the two strategies case, TFT and ALL-D, for K = 5 and three different 
migration rates, in terms of the average payoff a^. As explained in[B]afc = 1 means that ALL-D 
invades the entire population, whereas = 3 is achieved when TFT takes over. Fig. [6]^left) with 
m = is used for comparison only, as it shows the result predicted in[Bj I.e., three islands which 
have a frequency of TFT below the threshold value of 0.2 are taken over by agents playing ALL- 
D. The remaining two islands are invaded by TFT. This result changes in Fig. [6)^ middle), where 
we allow migration of m = 0.08 between islands. Then all islands become populated solely by 
TFT which is indicated by = 3. However, if we further increase the migration rate to m = 0.2, 
the opposite result occurs. In this case ALL-D invades all islands as shown in Fig. [Gfright) where 
Sfc = 1. These results suggest that there exists an optimal range for m that ensures the outbreak 
of cooperation, which will be further investigated in the following sections. 
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4.2 How migration influences cooperation 

We continue our discussion for the case of two strategies, TFT and ALL-D, and now assume 
that for K = 6 {K — 1) islands are initially dominated by stategy ALL-D, i.e. /j[^^(0) = for 
k G 2,..., 6, while the first island A; = 1 is initially populated by both TFT and ALL-D, i.e. 
< ^(0) < L We then vary the migration rate m for different initial frequencies /^^^(O) 
to study numerically under which condition the outbreak of cooperation on all islands can be 
obtained. The results are shown in Fig. [71 For very small m, the average payoff a just reflects 
the mix of the initial strategies. But, as the middle part of Fig. [7] shows, for a certain value of 
the initial frequency /|^^(0), we find a spike a = 3 at a particular value of m that indicates the 
outbreak of cooperation^ thanks to an optimal migration rate. One would assume that increasing 
the initial fraction of TFT would further boost cooperation but, as Fig. [7] (right) clearly shows, 
the "bandwidth" of optimal migration rates m is rather limited. Precisely, if /^^^(O) = 0.40 the 
bandwidth of optimal migration rates is 0.07 < m < 0.095 which is to be compared to the 
maximum bandwidth 0.07 < m < 0.155 for f[^\o) = 1. This makes sense because a large 
migration rate may have a contrary effect: While cooperative agents are "exported" to other 
islands, the same number of defective agents are "imported", which may reduce the fraction of 
TFT below the critical threshold. Hence, there is a nonmonotonous relation between m and the 
outbreak of cooperation. 




Figure 7: Increasing /{ (G = 0) leads to increasing bandwidth of m in which cooperation will 
emerge in the system. From left to right: (1) f[^\G = 0) = 0.31 (2) /f ^(G = 0) = 0.32 (3) 
/W(G' = 0) = 0.40 (4) f[^\G = 0) = 1.00. Parameters: Ug = A, K = 6 



4.3 Calculating the threshold frequency 

We are now interested in finding the minimum inital frequency f^^ (0) that will lead to invasion 
of cooperation on aW K — \ islands, when varying the migration rate m. We define this as the 
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threshold frequency, f^: 

Af(m^) = min/«(G = 0) (17) 

with fl^\G = 0) = 0y k £2,...,K ; f^^\G = oo) = iykeK 

For each f^, there exists a specific value of the migration rate E [0, 1] such that the outbreak 
of cooperation happens on all islands, denotes the the smallest value of the bandwidth of 
optimal migration rates shown in Fig. [T] I.e., for fixed rig and a given number of islands K 
with the initial conditions specified in eq. (|17p . there exist a tuple of critical values {f^,rn^) 
which determine the outbreak of cooperation. The outbreak shown in Fig. [71^ middle) for K = 6 
is observed for {f^,m^) = (0.32,0.07). This leads us to the question, how the threshold value 
and the critical migration rate change if we change the number of islands. 




K K 



Figure 8: (left) Threshold frequency f^ (black line) and relative effort f^/K (dashed line) 
dependent on the number of islands K. Symbols represent results of numerical calculations. 
Fitted values of eq. ^ Aq = 0.006, Ai = 4.798, A2 = 0.060. (right) Optimal migration rate 
dependent on the number of islands K. 

The results of exhaustive numerical calculations which had to search for the outbreak of coop- 
eration on the whole {K, f'^\{)),m) parameter space are shown in Fig. [SJ^left). Obviously, for 
K = \ the threshold frequency results from the analysis given in [Bj If the number of islands 
increases, f^'' (0) has to increase as well, in order to be able to "export" cooperating agents while 
still staying above the theshold. This increase, however, is nonlinear in K. Precisely, as can be 
verified in Fig. [8l 

fg, = K^Ao- {2AoAi) + K {AoAj + A2) (18) 

The values of the constants An depend implicitely also on the payoff matrix, eq. ([1]), which 
defines the comparative advantage of each strategy for reproduction and invasion, subsequently. 
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Plotting f^^/K we see that for 2 < K < 6, with every new island added the relative effort to 
invade the new one with cooperators decreases, while for K > 6 this effort increases quadrati- 
cally with K. However, the effort of invading other islands with cooperators always stays below 
the threshold of an isolated island, which is 0.2. This is a direct consequence of the combined 
processes of migration and reproduction. Newly arriving cooperative agents will reproduce on 
the other islands, this way reaching the threshold value the before the next generation is ready 
for migration. Hence, diversification of the reproduction sites for cooperating agents lower the 
relative effort for the outbreak of cooperation and makes it more favorable. 

As a second consequence of the nonlinear dependency, the export of cooperation from one island 
to — 1 other islands is only possible for a limited number of islands, because of the two effects 
already mentioned above. I.e. the "export" of cooperating agents and the "import" of defecting 
agents both decrease the fraction of cooperating agents below the threshold frequency. Hence, for 
the two strategy case discussed here, K = 8 already leads to the collapse of exporting cooperation. 

The critical migration rate dependent on the number of islands is shown in Fig. [Sl^right). 
We recall that this gives the minimal fraction of the population that has to migrate in order to 
allow the outbreak of cooperation on any island. If f[^\0) = /tf, it is at the same the maximum 
fraction to not let cooperation collapse back home, i.e. it is the optimal migration rate (see Fig. 
EJmiddle) 

Eventually, one can ask how the outbreak of cooperation [f^^,m^) is affected if instead of the 
two strategies TFT, ALL-D three or four strategies are considered. In general, the presence of 
both ALL-C and A-TFT favors the invasion of ALL-D at the end, whereas the presence of A- 
TFT favors the invasion of TFT - thus we can expect that the thresholds are slightly higher or 
lower in such cases. A more involved discussion is presented inlDl 

5 Conclusions 

With our investigations, we have contributed to the analysis of IPD games in two different 
domains: 

(i) For the meanfield limit, represented by a panmictic population where each agent interacts 
with N — 1 other agents Ug times, a quantitative investigation of the attractor basins was 
provided. Dependent on the initial mix of two cooperative (ALL-C, TFT) and two defective 
(ALL-D, A-TFT) strategies and the number of interactions Ug, we could derive the critical 
conditions (threshold frequencies) under which the outbreak of cooperation can be expected 
in a panmictic population. A detailed analysis of the two, three and four strategy cases 
helped to better understand the contribution of each strategy on the final outcome. 
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(ii) Using the "unpertubed" (or meanfield) limit as a reference point, the impact of migration 
on the outbreak of cooperation in a spatiahy distributed system could be quantified. We 
found that there is a bandwidth of optimal migration rates which lead to the induction 
(or "export") of cooperation to a number of islands which otherwise would converge to 
defection. We were able to determine the critical conditions f^i^,.,m^ which guarantee the 
outbreak of cooperation in a worst case scenario. Remarkably, the relative effort fthr/K to 
export cooperation to defecting islands was decreasing with K in some range and always 
stayed below the critical value for an isolated island. I.e. effectively it became easier to turn 
defecting into cooperating islands, provided the optimal migration rate was reached. 

It is important to note that the outbreak of cooperation is not enforced by a maximal migration 
rate but, as stated above, by a small range of optimal migration rates. This is because migration, 
different from other approaches, is not seen as an unidirectional dynamics, where agents just 
move to a "better" place. Instead, our model is based on the assumption that it is a bidirectional 
process, which we deem a more realistic assumptions when considering a dynamics over many 
generations. In fact, agents, or their offsprings, which have obtained certain skills or wealth at 
their immigration country, quite often decide to start new buisness back home at their origin 
country - because their new capabilities give them a considerable advantage there, while it is 
only a marginal advantage in their immigration country. This assumption is in line with many 
policies about immigration/education of foreigners in a country, to give indirect support for 
development and to allow future business with those countries, where some agents with positive 
experience have resettled. At the end, as we have shown in our paper, it remarkably helps to 
spread "cooperation" globally, if the number of breeding places for such a strategy is increased 
(above a minimum threshold). 
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A Calculating the average payoff 

In order to illustrate eq. 1^, let us look at two agents i and j. If both of them play strategy z^^\ 
TFT, then, during the Ug different "iterations" , their choices are as follows: 



(19) 
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Consequently, the average payoff for both ag 
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i and j is af'^'' - 
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which leads to an average payoff for agent i 


of 


2) = [S + (n. 


— l)P]/ng, while the 
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payoff for agent j is a 



(21) 



[T + {ng-l)P]/ng. 



If agent i playing A-TFT interacts with agent j playing ALL-D, the series of choices reads: 



payoff a^^"^^ 
{A - TFT) 
[ALL - D) 
payoff a^-^"^^ 



1 

P 
D 



2 

S 
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,(42) 



n 



9 
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(21) 



That means the average payoff for agent i is a- = [P + (ug — l)S]/ng, while the average payoff 



for agent j is a 



(24) 



[P + (ug — l)T]/ng. The remaining cases can be explained similarly. Only 
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the three more complex cases are given below, a-^^^ follows from: 



payoff af''^^ 
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{A - TFT) 
payoff aj^^^ 
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(22) 



Here, the average payoff of agent i results from a repeated payoff series S PT R, and the value 
of Ug defines when this series is truncated. Similarly, the series for a^^^^ is T P S R. Thus the 
first term of af""^^ , eq. Q calculates all completed payoff series (ug div 4) , while the second term 
calculates the remaining payoffs [ug rem 4) . 

For a , we find eventually: 



payoff a'l^^^ 
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(A - TFT) 
payoff a^-^^^ 
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Here, the period of the payoff series is just 2 instead of 4, and the expression for a, 
eq. Q follows accordingly. 



(44) 
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B Two strategies: TFT and ALL-D 



We assume that the population consists of only two strategies: z^^^ =TFT and z^^^ =ALL-D, eq. 
([3]). This is an interesting combination since defection is known to be the only equilibrium in an 
one-shot PD game, while TFT fares well in a repeated PD interaction, given there is a critical 
number of encounters (discussed below). 

The average payoff a[^'^^ received by agent i playing strategy z^'^-' {q G 1,2) results from eq. ([8]). 
Choosing rig = 4 we find: 



a(ii) a(i2) 
a(2i) a(22) 



3.0 0.75 
2.0 1.0 



Applying eqs. (jlip . we find for the total average payoff: 



(24) 



(25) 
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On the other hand, it follows from the stationary condition ()12p : 

a /W - f a(*i) /(I) + a(*2) f{2)\ ^ g (i = 1, 2) 



(2) 



(26) 



1 



The combined eqs. (j25p . (j26p have to be solved simultaneously for the possible stationary fre- 
quencies /(I), As the result we find: 



/(I) 
/(I) 

•'thr 



0, = 

1, /(^) = 

0-2, 



1 (stable) 
(stable) 
= 0.8 (for Ug 



(27) 



4) (unstable) 



Solution (i) implies that either strategy ALL-D or TFT invades the population completely, which 
are the two stable attractors of the system in the case of only two strategies. On the other hand, 
solution (ii) describes the coexistence of the two strategies with different frequencies within the 
total population. In the given case, it is is an unstable point attractor that separates the basins 
of two stable attractors, and therefore acts as a separatrix here. Le., for an initial frequency 
/(^)(0) < 0.2 of strategy z^^^ =TFT, the dynamics of the system will converge into a stationary 
state that is entirely dominated by strategy ALL-D (where the payoff per agent is a = P = 1), 
whereas in the opposite case the population will entirely adopt strategy TFT (average payoff 
a = R = 3). 

(s) 

The threshold value /^.j^^ strongly decreases with the number of encounters Ug, as shown in Fig. 
[2] for the case of four strategies. This raises the question about the critical Ug for which TFT 
could invade the whole population. Let us compare the average payoffs of the strategies z^^'^ = 
TFT and = ALL-D: 



,(1) =„(11)J(1)+„(12)^(2) 
j(2) ^^(12) ^(1) ^ ^(22) ^(2) 



(28) 



The respective values of a^'"''-' can be calculated from eq. ([8]) for different numbers of Ug. We find 
that only the elements a^^"^^ and a^^^-* change with follows: 



Ug 


1 


2 


3 






0.0 


0.5 


0.677 




a(2i) 


5.0 


3.0 


2.333 





(29) 



With a(i^) = 3.0 and 0(^2) = l.Q, we find for = 1 that a^^i) > a(^i),a(22) > 0(^2)^ Tj^jg implies 
that a^2) > a^^^ for all possible initial frequencies of the two strategies in the population. Thus, 
from the dynamics of eq. ([5]) (fitness-proportional selection) the extinction of the strategy TFT 
results, in agreement with the known results. For Ug = 2 we find a^^^^ = a^^^\ a^'^'^^ > a^^'^\ 
which again implies that a'-^-* > a*-^^ for all possible initial frequencies of the two strategies in 
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the population, i.e. the extinction of strategy TFT. However, for Ug = 3, we find a^^^^ < a^^^\ 
a^^^-* > a^^'^^ . Thus, there exist some initial frequencies of the two strategies for which a^^^ > a^^^ 
results. I.e., we can conclude that for the given payoff matrix n*^'' = 3 is the threshold value for 
the number of interactions between each two agents that may lead to the invasion of cooperation 
in the whole population. 

C Three strategies: TFT, ALL-D and ALL-C 

When strategy z^^^ = ALL-C is added to the previous strategies z^^^ = TFT and z^'^^ = ALL-D, 
agents playing ALL-D will benefit from agents playing ALL-C. Therefore, if the frequency of 
agents playing ALL-C is increased in the initial population, it can be expected that the attractor 
basin of the TFT strategy will shrink, while the attractor basin of the ALL-D strategy will grow, 
compared to the case of two strategies, discussed in[Bl The stationary frequencies have now to 
be calculated from the following set of coupled equations that result from eqs. (jlip and the 
stationary condition (fT2|l : 

s=l s=l \r=l / 

fil) + fi2) + ;(3) 

The matrix elements a^^"^^ can be calculated from eq. ([8]). With = 4 we find the following 
stationary solutions: 

(i) /(I) = 1, /(2) = 0, = (stable) 
/(I) = 0, = 1, /(3) = (stable) 

/(I) = 0, = 0, = 1 (unstable) (31) 

(ii) = 0.2, fill = 0.8, = (for ng = 4) (unstable) 
/(I) = 1.0 - /(3) (0 < < 1), /(2) = (stable) 

Again, the solutions (i) imply that either strategy TFT or ALL-D or ALL-C invades the entire 
population. We note that only the first two solutions are stable ones, while the last point-attractor 
/(^) = 1, /(^) = /(^^ = is an unstable one, because any small pertubation (i.e. the invasion of 
one defecting agent) will transfer the cooperating system into a defecting one. 

The first of the solutions (ii) describing coexisting strategies is already known from the investi- 
gation in[B]to be an unstable one. In the absense of the third strategy, it defines the separatrix 
point. The second solution (ii) however is a stable one, indicating that both agents playing TFT 



(i = 1,2,3) 



(30) 
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and ALL-C can coexist in the panmictic population. Note that there is neither a stable nor an 
unstable coexistence of all three strategies. 

The separatix that divides the different attractor basins is now a line in a two-dimensional space 
of the initial frequencies. But different from [B] the stationary solutions of eq. ([3T]) do not give 
further information about the separatices. In order to calculate the different attractor basins, we 
therefore have numerically solved eq. ([5]) for the full range of initial frequencies: < f^^\0) < 1, 
< /(2)(0) < 1, /(^^O) = 1 - /(^)(0) - f^^\0), and have evaluated the average payoff in the 
stationary limit. If a = P, obviously the whole population has adopted strategy z^"^^ = ALL- 
D. Similarly, if a = R and /^^^ = 1, the whole population has adopted strategy z^^^ = TFT. 
However, if a = R and f^^^ < 1, then there is a coexistence of agents playing strategy z^^^ = 
TFT and = ALL-C. 

The attractor basins are shown in Fig. [9] for two different values of = 4 and — > oo. In the 
latter case, a^^^^ = a^"^^^ = 1 results for the payoffs in eq. ([5]). For = 4 we can distinguish 
between three different regions in Fig.[9l Region A denotes the range of initial frequencies /^*^(0) 
that lead to the adoption of the z^"^^ = ALL-D strategy in the whole population, region B denotes 
the range of initial frequencies that lead to the adoption of the z^^^ = TFT strategy in the whole 
population, while region C denotes the range of initial frequencies that lead to the coexistence 
of both = TFT and z^^) = ALL-C strategies. 




Figure 9: Attractor basins, i.e. range of initial frequencies /^*^(0) that lead to a particluar stable 
solution, eq. (|3ip . A: adoption of z*-^-* = ALL-D strategy in the whole population, B: adoption 
of z^-^^ = TFT strategy in the whole population, C: coexistence of both z^^^ = TFT and z^^^ = 
ALL-C strategies. /(3)(0) = 1 - /(^)(0) - f^'^\0)- (left) Ug = 4, (right) Ug oo. 

Since regions B and C both describe the adoption of cooperating strategies in the population, 
the most interesting line in Fig. [9] is the separatrix between region A (all defectors) and region 
B (all cooperators). We can easily interpret this line based on our previous analysis of the two- 
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strategy case, [Bj The diagonal in Fig. [9] represents /^^^ + /(^) = 1, i.e. a population with only 
two strategies, z^'^^ = TFT and z^"^^ = ALL-D. Thus the separatrix line between regions A and 
B starts from the separatrix point /^^^ = 0.2, /^^^ = 0.8, /^^^ = 0, eq. ([3T]) . Further below the 
diagonal, the frequency /^^^ of the strategy z^^^ = ALL-C increases in the initial population, 
which in turn increases the threshold frequency f^^' necessary for the invasion of the z(^) = TFT 
strategy. In a certain range of frequencies, the separatrix line between defection and cooperation 
can be described by a linear relation, found numerically as: 



/ 



(2) 



6.1538/(^) + 2.0308 for 0.2 < < 0.31 



(1) 



(32) 



However, we notice that the separatrix line between regions A and B never hits the x-axis. For 
very low values of /'•^^(O), i.e. close to the x-axis, it makes a sharp turn towards the origin. This 
means that for a vanishing initial frequency of ALL-D there will be no route to the respective 
attractor region A, which is obviously correct. 

The influence of the parameter Ug is shown by comparing the left (ug = 4) and the right part 
{ug = oo) of Fig. [9j In the latter case the attractor basin A (exclusive domination of strategy 
2;(2) = ALL-D) becomes very small. In order to further quantify the influence of ng on the 
dominating strategies in the stationary limit, we have also calculated the relative size of each 
attractor basin. If F^, Fb and Fc denote the area of the regions A, B and C in Fig. [9l the 
relative sizes are defined as follows: 



Fa 



; b 



Fb 



Fc 



Fb + Fc 



tot 



Ftot Ftot Ftot Ftot 
The results are shown in Fig. [10] for the two different values of Ug. 



Fa + Fb + Fc 



(33) 




Figure 10: Relative size, eq. (|33p of the attractor basins shown in Fig. [9j The left bars (shaded 
area) refer to = 4 (Fig. Elleft), while the right bars (black area) refer to — )• oo (Fig. 
right). Thus, the change indicates the influence of Ug on the size of the attractor basins. 



In Fig. [TOl d, eq. ([33]) denotes the relative size of the cooperation attractor basin resulting from 
both solutions, domination of z^^'' = TFT and coexistence of z^^'' = TFT and z^^^ = ALL-C. 
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As we see, for Ug = 4, the cooperative basin d only has about the same size as the defection 
attractor basin a, that mean that about half of the possible initial conditions will lead to a 
population of defecting agents, at the end. Only for Ug ^ oo the size of the defection basin 
becomes insignificant as compared to the cooperative basin. This again explains the role of Ug 
in influencing cooperation. 



D Threshold frequencies for three and four strategies 

In Section [4.31 we computed the critical conditions for the outbreak of cooperation {f^,m^) in 
the presence of only two strategies, ALL-D and TFT. If we consider the two additional strategies 
ALL-C and A-TFT, eq. Q, these critical conditions change dependent on the initial values of 
the four strategies and their distribution on the different islands. Thus, instead of a complete 
investigations we only discuss the following sample configurations with fff\o) = 1: 



(a) three strategies, TFT, ALL-D, ALL-C. /f^(0) > 0,/|"^(0) = 1 - /j'^(O), f^^'{0) = 1 



A3), 



fP{0) = 1 with A: > 3. 



(b) four strategies, TFT, ALL-D, ALL-C, A-TFT. f[^\o) > 0,f[^'{0) = l-f['>{0), /^^^(O) 



1, /3^'^ = 1, /r(0) = lwithA;>4. 



p(2) 



These initial conditions imply that K > 3. The results of extensive calculations of the relative 
effort f^j./K and the critical migration rate are shown in Fig. [11] and can be compared to 
the two strategy case, eq. ()17p . 
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Figure 11: (left) Threshold versus number of islands K. (right) Migration rate m versus 
number of islands K. Migration can only promote cooperation if the number of islands is < 7 



27^ 



F. Schweitzer, L. Behera: 
Optimal migration promotes the outbreak of cooperation in hetergeneous populations 

ACS - Advances in Complex Systems (2012) 



Again, we notice that the relative effort to invade other islands by agents playing TFT is non- 
monotonously dependent on K and always stays below the threshold values observed without 
migration, as given in Sect. 12. 4| ICl Regarding the impact of the different strategies on the 
outbreak of cooperation, we see that in the presence of ALL-C ALL-D benefits more than TFT 
in terms of payoff which raises the threshold frequency. However, adding A-TFT benefits ALL- 
D less than adding ALL-C, which lowers the threshold frequency. This explains why the curve 
for the four strategy case is in between the curves corresponding to two and three strategies. 
From Fig. [TlT right) the optimal migration rate is found to be almost constant for three and four 
strategies in contrast to the two strategy case. 
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